Evidence summaries: the evolution of a rapid review approach

<p>Abstract</p> <p>Background</p> <p>Rapid reviews have emerged as a streamlined approach to synthesizing evidence - typically for informing emergent decisions faced by decision makers in health care settings. Although there is growing use of rapid review 'methods', and proliferation of rapid review products, there is a dearth of published literature on rapid review methodology. This paper outlines our experience with rapidly producing, publishing and disseminating evidence summaries in the context of our Knowledge to Action (KTA) research program.</p> <p>Methods</p> <p>The KTA research program is a two-year project designed to develop and assess the impact of a regional knowledge infrastructure that supports evidence-informed decision making by regional managers and stakeholders. As part of this program, we have developed evidence summaries - our form of rapid review - which have come to be a flagship component of this project. Our eight-step approach for producing evidence summaries has been developed iteratively, based on evidence (where available), experience and knowledge user feedback. The aim of our evidence summary approach is to deliver quality evidence that is both timely and user-friendly.</p> <p>Results</p> <p>From November 2009 to March 2011 we have produced 11 evidence summaries on a diverse range of questions identified by our knowledge users. Topic areas have included questions of clinical effectiveness to questions on health systems and/or health services. Knowledge users have reported evidence summaries to be of high value in informing their decisions and initiatives. We continue to experiment with incorporating more of the established methods of systematic reviews, while maintaining our capacity to deliver a final product in a timely manner.</p> <p>Conclusions</p> <p>The evolution of the KTA rapid review evidence summaries has been a positive one. We have developed an approach that appears to be addressing a need by knowledge users for timely, user-friendly, and trustworthy evidence and have transparently reported these methods here for the wider rapid review and scientific community.</p

The use of rapid review methods in health technology assessments: 3 case studies.

BACKGROUND: Rapid reviews are of increasing importance within health technology assessment due to time and resource constraints. There are many rapid review methods available although there is little guidance as to the most suitable methods. We present three case studies employing differing methods to suit the evidence base for each review and outline some issues to consider when selecting an appropriate method. METHODS: Three recently completed systematic review short reports produced for the UK National Institute for Health Research were examined. Different approaches to rapid review methods were used in the three reports which were undertaken to inform the commissioning of services within the NHS and to inform future trial design. We describe the methods used, the reasoning behind the choice of methods and explore the strengths and weaknesses of each method. RESULTS: Rapid review methods were chosen to meet the needs of the review and each review had distinctly different challenges such as heterogeneity in terms of populations, interventions, comparators and outcome measures (PICO) and/or large numbers of relevant trials. All reviews included at least 10 randomised controlled trials (RCTs), each with numerous included outcomes. For the first case study (sexual health interventions), very diverse studies in terms of PICO were included. P-values and summary information only were presented due to substantial heterogeneity between studies and outcomes measured. For the second case study (premature ejaculation treatments), there were over 100 RCTs but also several existing systematic reviews. Data for meta-analyses were extracted directly from existing systematic reviews with new RCT data added where available. For the final case study (cannabis cessation therapies), studies included a wide range of interventions and considerable variation in study populations and outcomes. A brief summary of the key findings for each study was presented and narrative synthesis used to summarise results for each pair of interventions compared. CONCLUSIONS: Rapid review methods need to be chosen to meet both the nature of the evidence base of a review and the challenges presented by the included studies. Appropriate methods should be chosen after an assessment of the evidence base

Maximising the availability and use of high quality evidence for policymaking:Collaborative, targeted and efficient evidence reviews

Abstract A number of barriers have been identified to getting evidence into policy. In particular, a lack of policy relevance and lack of timeliness have been identified as causing tension between researchers and policy makers. Rapid reviews are used increasingly as an approach to address timeliness, however, there is a lack of consensus on the most effective review methods and they do not necessarily address the need of policy makers. In the course of our work with the Scottish Government’s Review of maternity and neonatal services we developed a new approach to evidence synthesis, which this paper will describe. We developed a standardised approach to produce collaborative, targeted and efficient evidence reviews for policy making. This approach aimed to ensure the reviews were policy relevant, high quality and up-to-date, and which were presented in a consistent, transparent, and easy to access format. The approach involved the following stages: 1) establishing a review team with expertise both in the topic and in systematic reviewing, 2) clarifying the review questions with policy makers and subject experts (i.e., health professionals, service user representatives, researchers) who acted as review sponsors, 3) developing review protocols to systematically identify quantitative and qualitative review-level evidence on effectiveness, sustainability and acceptability; if review level evidence was not available, primary studies were sought, 4) agreeing a framework to structure the analysis of the reviews around a consistent set of key concepts and outcomes; in this case a published framework for maternal and newborn care was used, 5) developing an iterative process between policy makers, reviewers and review sponsors, 6) rapid searches and retrieval of literature, 7) analysis of identified literature which was mapped to the framework and included review sponsor input, 8) production of recommendations mapped to the agreed framework and presented as ‘summary topsheets’ in a consistent and easy to read format. Our approach has drawn on different components of pre-existing rapid review methodology to provide a rigorous and pragmatic approach to rapid evidence synthesis. Additionally, the use of a framework to map the evidence helped structure the review questions, expedited the analysis and provided a consistent template for recommendations, which took into account the policy context. We therefore propose that our approach (described in this paper) can be described as producing collaborative, targeted and efficient evidence reviews for policy makers

"Best fit" framework synthesis: refining the method

Background Following publication of the first worked example of the “best fit” method of evidence synthesis for the systematic review of qualitative evidence in this journal, the originators of the method identified a need to specify more fully some aspects of this particular derivative of framework synthesis. Methods and Results We therefore present a second such worked example in which all techniques are defined and explained, and their appropriateness is assessed. Specified features of the method include the development of new techniques to identify theories in a systematic manner; the creation of an a priori framework for the synthesis; and the “testing” of the synthesis. An innovative combination of existing methods of quality assessment, analysis and synthesis is used to complete the process. This second worked example was a qualitative evidence synthesis of employees’ views of workplace smoking cessation interventions, in which the “best fit” method was found to be practical and fit for purpose. Conclusions The method is suited to producing context-specific conceptual models for describing or explaining the decision-making and health behaviours of patients and other groups. It offers a pragmatic means of conducting rapid qualitative evidence synthesis and generating programme theories relating to intervention effectiveness, which might be of relevance both to researchers and policy-makers

Assessing the format and content of journal published and non-journal published rapid review reports : A comparative study

BACKGROUND: As production of rapid reviews (RRs) increases in healthcare, knowing how to efficiently convey RR evidence to various end-users is important given they are often intended to directly inform decision-making. Little is known about how often RRs are produced in the published or unpublished domains, and what and how information is structured. OBJECTIVES: To compare and contrast report format and content features of journal-published (JP) and non-journal published (NJP) RRs. METHODS: JP RRs were identified from key databases, and NJP RRs were identified from a grey literature search of 148 RR producing organizations and were sampled proportionate to cluster size by organization and product type to match the JP RR group. We extracted and formally compared 'how' (i.e., visual arrangement) and 'what' information was presented. RESULTS: We identified 103 RRs (52 JP and 51 NJP) from 2016. A higher percentage of certain features were observed in JP RRs compared to NJP RRs (e.g., reporting authors; use of a traditional journal article structure; section headers including abstract, methods, discussion, conclusions, acknowledgments, conflict of interests, and author contributions; and use of figures (e.g., Study Flow Diagram) in the main document). For NJP RRs, a higher percentage of features were observed (e.g., use non-traditional report structures; bannering of executive summary sections and appendices; use of typographic cues; and including outcome tables). NJP RRs were more than double in length versus JP RRs. Including key messages was uncommon in both groups. CONCLUSIONS: This comparative study highlights differences between JP and NJP RRs. Both groups may benefit from better use of plain language, and more clear and concise design. Alternative innovative formats and end-user preferences for content and layout should be studied further with thought given to other considerations to ensure better packaging of RR results to facilitate uptake into policy and practice. STUDY REGISTRATION: The full protocol is available at: https://osf.io/29xvk/